Human and automatic speech recognition in the presence of speech-intrinsic variations

Despite several decades of research, automatic speech recognition (ASR) lacks the performance achieved by human listeners. One of the major challenges in ASR is to cope with the immense variability of spoken language, which can be categorized into extrinsic sources (e.g., additive noise) and intrinsic factors (such as speaking rate, style, effort, dialect, and accent). What can we learn from the biological blueprint, and which cues important in human speech recognition (HSR) should be considered to improve ASR performance? The scope of this thesis is to answer these questions by comparing the HSR and ASR performance and based on these results to suggest an alternative way of feature extraction to improve ASR. The comparison is based on the Oldenburg Logatome Corpus, which is a database that contains simple nonsense words consisting of phoneme triplets and which covers the intrinsic variations mentioned above. The man-machine-gap in terms of the signal-to-noise ratio (SNR) was estimated to be 15 dB, i.e., the masking level in ASR has to be lowered by 15 dB to achieve the same performance as human listeners. The contributions to this gap could be attributed to the individual processing steps of the ASR system: The information loss caused by the feature extraction resulted in an SNR-equivalent information loss of 10 dB, while suboptimal classification accounted for the remaining 5 dB of the overall gap. Moreover, the analysis of intrinsic variations showed that human listeners are superior to ASR systems in exploiting temporal cues. These findings motivated the use of spectro-temporal Gabor features in ASR, which were found to exhibit increased robustness against a wide range of noise types. In the presence of intrinisic variations of speech, Gabor features increase the overall performance regarding several factors (such as speaking effort and style), which suggests to incorporate both spectro-temporal and temporal cues in future ASR systems.